@techsavvyash please add unit + e2e. Need this to be tested on every commit.

I tested this on gitpod, but I am not able to see the values being updated and getting these errors instead. Can you also share the ingest files that you are using.

Have we switched to using PG_POOL ? @techsavvyash

Have we switched to using PG_POOL ? @techsavvyash

yes! The changes in this PR are ported from #101

I tested this on gitpod, but I am not able to see the values being updated and getting these errors instead. Can you also share the ingest files that you are using.

DMed on discord

@techsavvyash please add unit + e2e. Need this to be tested on every commit.

added.

I tested this on gitpod, but I am not able to see the values being updated and getting these errors instead. Can you also share the ingest files that you are using.

checking into this, meanwhile you can refer this video: https://drive.google.com/file/d/1YOjHJCkuHzb1ttI1B_3aLSE6bRcv2waZ/view?usp=share_link

@tushar5526 checked on gitpod, not able to reproduce the error mentioned, can you share the steps you followed.
Here is what I did:

1. Open the PR in gitpod
2. create a top level .env at the root of the project using the given .env.sample.
3. run docker-compose up -d
4. create a .env in impl/c-qube with the following format

DATABASE_URL="postgres://timescaledb:postgrespassword@localhost:5432/postgres?sslmode=disable"
DB_USERNAME="timescaledb"
DB_HOST="localhost"
DB_NAME="postgres"
DB_PASSWORD="postgrespassword"
DB_PORT="5432"

5. install dependencies inside impl/c-qube using yarn
6. run yarn cli ingest
7. run yarn cli ingest-data

and everything worked fine for me.
Attaching a screenshot for reference, check the bottom left to confirm that its the gitpod environment.

@tushar5526 please review this.

LGTM, great work @techsavvyash! Thanks

Pull Request Test Coverage Report for Build 6693125896

• 0 of 0 changed or added relevant lines in 0 files are covered.
• No unchanged relevant lines lost coverage.
• Overall coverage remained the same at 0.0%

Totals
Change from base Build 6080431350:	0.0%
Covered Lines:	0
Relevant Lines:	0

💛 - Coveralls

Works great! Thanks @techsavvyash!

I still see some errors while testing @techsavvyash

I still see some errors while testing @techsavvyash

I am getting no error of this sort on my local with the same ingest folder. Can you share reproduction steps?

Error: ENOENT: no such file or directory, stat './ingest/programs/diksha/linkedqrcount-event.data.csv'
    at Object.stat (node:internal/fs/sync:58:25)
    at Object.statSync (node:fs:1669:17)
    at createCompoundDatasetDataToBeInserted (/workspace/cQube-ingestion/impl/c-qube/src/services/csv-adapter/parser/dataset/dataset-grammar.helper.ts:110:20)

not able to reproduce, please share reproduction steps.

The error reported is not being reproduced on gitpod as well for me.

Tested again, works good. Thanks!